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LIVE STREAMING BROADCASTING METHOD, LIVE STREAMING 
BROADCASTING APPARATUS, LIVE STREAMING BROADCASTING SYSTEM, 
PROGRAM, RECORDING MEDIUM, BROADCASTING METHOD AND 
BROADCASTING APPARATUS 

TECHNICAL FIELD 

[0001] The present invention relates to a live streaming broadcasting mehedmethod, a 
live streaming broadcasting apparatus, a live streaming broadcasting system, 
programs, a recording medium, a broadcasting method and broadcasting 
apparatus. 

BACKGROUND ART 
[0002] There has been carried out conventionally an internet live broadcasting 
broadcast for broadcasting images (pictures) and voices to clients through a 
network such as the i nternet, that is, a live streamin g broadcastin gb roadcast . 
[0003] In order tbat-foLclients to audit -listen to l ive streamin g broadcastin g broadcasts , 
for example, a^browser is -may be s tarted by a -an auditory t erminal for auditory 

to get access to a home page of a broadcasting presenter^ (a broadcaster), 

and data of the broadcasting contents Broadcasting content data i s received by 
the auditor/ termina l for auditory . The data received by the auditory t erminal 
for auditory i s converted into a streaming file by the decoding process in a 
streaming player (including a streaming decoder) incorporated in the terminal 
for auditory in advance so that an image etrt-ef -from the broadcasting contents 
broadcast content i s displayed on the-a_display screen of the auditorv terminal 
for auditory , and the voice is output from a speaker. Thereby, the clients are 
able to atidi Histen to the broadcasting contents. 
[0004] It is to be noted that the auditorv terminal for auditory i s can be. for example, a 
general purpose PC (Personal Computer). Further, the streaming player is a 
streaming player incorporated into a general purpose browser, or an exclusive 
use streaming player, both of which are constructed within the auditorv 
terminal for auditory by installing a program (software) on the auditorv 
tenninal for auditory . 



[0005] On the other hand, in order that -to allow t he broadcaster performs to initiate the 
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live streaming broadcnstin g broadcasL a broadcasting program is started by a 
broadcasting terminal, while voice data afe~is input into the broadcasting 
terminal from, for example, camera image data and-grik emicrophone . And, 
these -This data efe-is subjected to encode-processing in accordance with the 
broadcasting program started to allow the data to be o utput them t o the 
network. 

[0006] It is noted that the broadcasting terminal is, for example, a general purpose PC, 
Andand, the broadcasting program is a general purpose program (software) 
including a function as a streaming encode r functionality . 

[0007] FIG 14 shows q flow of i s a flow chart showing t he live streaming broadcasting 
as described above. 

[0008] As shown in FIG 14, on the broadcaster side, image data (animations) from a 
camera 101 and voice data from a grifee -microphone 102 are encode-processed 
and converted into a streaming file, which is continuously output to a network 
104. Broadcasting data to be output are input into a streaming server 106. 
[0009] Further, in a -an auditory terminal for auditory^ 105 on the client side, a browser 
105a is started so that broadcasting data from the broadcaster ore i s 
continuously received through a network 104 from a streaming server 106, the 
received broadcasting data afe— is decode-processed by a streaming player 
(streaming decoder) 105b within the auditory terminal for auditory 105 to 
continuously carry out image display and voice output. Thereby, on the auditor 
listener s ide, broadcasting through the network 104 can be audited e xperience 
(on live) at-in real time. 
_[0010] As the technique in conn e ction with the live streaming broadcasting as 

described, there is a technique of Patent Reference 1, for example. 
[0011] Incidentally, in the In_conventional live streaming broadcasting for carrying out 
editing mainly using a single apparatus, for example, such as a general purpose 
PC, the broadcasting mode for broadcasting images and voices sound w ithout 
applying any process thereto is a mainstream, and the expression thereof 
involved i nvolves a great difference as compared with a-radio broadcasting. 
That is, conventionally, it has been impossible, for example, to synthesize a 
plurality of camera images, to insert a telop, and to perform processes for 
synthesizing video images (such as alpha blend process, lay over laying 
process). 

[0012] Concretely, for oxamplo, according According to the editing function provided 
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in ""WindowsMedia Encode r ^ goods namcj j which is a software for the 

streaming encode encoding made by Microsoft Inc., only one camera source is 
selected, and a plurality of camera images cannot be displayed simultaneously. 
[0013] Further, in case of with respect to processing images, for example, a s disclosed 
in Patent Reference 1, there i s merely present such that typically, separate 
images are to be merely d isplayed on a plurality of display regions , respectively, - 
within the display screen on the clients side. 

[0014] In order thot cinventionally, for exampl e , synthesizing to synthesize a plurality 
of camera images, inserting insert a telop, and performin g- perform processes 
for synthesizing video images (such as alpha blend process, lay over laying 
process) . are carried out , it was historically necessary to use a broadcasting 
system 200 provided with many broadcasting apparatuses in addition to 
P^O OPC 201, as shown in FIG 15, for example. 

[0015] That is, the broadcasting system 200 shown in FIG 15 is provided, for editing 
images, with for example, P€a&jr -PC 201 having display data such as a telop 
stored therein, a down converter 202, a plurality of video decks 203 for 
regenerating a video tape, a switcher 204 for selecting one out of these image 
data, a confirming monitor 205, a plurality of cameras 206, a switcher 207 for 
selecting one out of image data from the plurality of cameras 206, a confirming 
monitor 208, a video mixer (which performs alpha blend process, lay 
overlaying process, etc) for synthesizing image data from the switches 204, 
207), and a monitor 210 for confirming image data after it has been s ynthesized 
by the video mixer 209. 

[0016] Further, for editing voices, there ftfe-is.provide d, for example, with a sampler 
211 for sampling effect sound, an effecter for applying effect process to effect 
sound, a naage -microphone 213, a player 214 such as a CD player, MIDI 
apparatus 215 for regenerating ^ MIDI file, voice apparatus 216 for 
line-inputting voice data, a mixer 217 for mixing these -the v oice data, and a 
monitor 218 for monitoring the v oice data after mixing by the mixer 217. 

[0017] Further, the PC 220 is provided with a video capture 221 for receiving image 
data from the video mixer 2097^, a sound card 222 for receiving voice data from 
the mixer 217, and a stream encoder (streaming encoder) 223 fii^for 
encode-processing voice data from the sound card 222 and image data from the 
video capture 221 into a^streaming broadcasting to output them b roadcast for 
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outputting to the network 104. 
Patent Reference 1: Japanese Patent Laid Open No. 2003 125339 Publication 

DISCLOSURE OF INVENION SUMMARY OF THE INVENTION 

Problem to be solved by the Invention 
[0018] In the case of the conventional live streaming broadcasting for performing 

editing mainly using a single apparatus such as a general purpose PC, there was 

a great difforont difference in its image and voice from radio broadcasting, as 

described above. 

[0019] Further, concretely, for example, i n the case of using [Windows Media 

Encode r Fgoods name J J, for image-switching between cameras, it is necessary 

to take the procedure for starting a separate camera after finishing operation of 
the camera selected originally , thus posing a problem that > This is 
problematic because it t akes time for switching. 

[0020] Further, in the case of using a broadcasting system 200 provided with many 
broadcasting devices as shown in FIG 15, there needs costs for motorials, and it 
the materials are costlv and it t akes time for installation of materials as wcH -as 
connection to install and connect the materials . In addition, in case of the 
broadcasting system 200, there poses a problem discussed below. 

[0021] FIG 16 is a flowchart showing a flow of processes carried out particularly in a 
switcher 207 and a video mixer 209, out of various broadcasting devices shown 
in FIG 15. 

[0022] As shown in FIG 16, in the switcher 207, image data af=e-4s^input from a 
plurality of cameras 206 (Step SlOl), D/A conversion is carried out with 
respect to these image data (Step S402), and subsequently, image data from a 
camera 21 that is selected by operation of a broadcaster out of the image data 
are selected (Step S103). Then, the selected image data is subjected to D/A 
conversion (Step S104) to output it from the switcher (Step S105). 

[0023] Further, in the video mixer 209, image data from the switcher 207 dfe-is 
respectively input (S106), which are subjected to A/D conversion with respect 
to the image data (Step S107). Then, the image data after A/D conversion are 
synthesized (Step S108), and the image data after synthesized are subjected to 
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D/A cx)nversion to output it to PC 220 from the video mixer 209. 

[0024] That is, in the case of the broadcasting system 200, since the synthesizing 
process (Step S104) is carried out, it is necessary, as shown in FIG 16, to carry 
out output and input of image data (Step S105 and Step S106), and it is also 
necessary to repeat A/D conversion (Step S102 and Step S107) and D/A 
conversion (Step S104 and Step S109), resulting in a lot of wastes in process. 
Moreover, input and output, and D/A conversion are repeated, thus posing a 
problem of increasing possibility that noses are produced in image data. 

[0025] Further, for inserting a telop in the conventional live streaming broadcasting, it 
is necessary to prepare display data for a telop in advance to store it in PC201, 
which is troublesome, and failing to correspond thereto in the case where a 
telop becomes necessary suddenly. 

[0026] Further, in the conventional live streaming broadcasting, it is merely that one 
dimensional broadcasting from a single streaming server 106 can be audited. 
Therefore, it was not possible to audit multi-dimensional broadcasting from a 
plurality of streaming servers 106. 

[0027] In the live streaming broadcasting, there is a problem that for the convenience' 
sake of data amount that can be processed, it is difficult to use excessively large 
image data for broadcasting. Therefore, it has been desired to be data amount 
processed as small as possible, as well as broadcasting contents excellent in 
expression. 

[0028] Furthermore, also in various broadcastings not limited to the live streaming 
broadcasting, a broadcasting method in novel expression that not found in prior 
art has been desired. 

[0029] The present invention has been accomplished in order to solve the problems as 
noted above, and has its object to provide a live streaming broadcasting method, 
a live streaming broadcasting apparatus, a live streaming broadcasting system, 
programs, a recording medium, a i>roadcasting method and a broadcasting 
apparatus, which realize broadcasting in high expression at low cost, or in 
novel expression not obtained so far. 

Means for solving the problem 
[0030] For solving the aforesaid problems, the live streaming broadcasting method of 
the present invention is a live streaming broadcasting method for carrying out a 
live broadcasting through a network, characterized in that, while inputting a 
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plurality of camera image data, synthesized image data obtained by 
synthesizing process for synthesizing a plurality of camera image data during 
inputting are output for auditory by clients. 
[0031] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting 
broadcast through a network, characterized in that, such that w hile receiving 
the other live streamin g broadcastin g broadcast, image data of the live 
streaming broodcasting broadcast d uring receiving afe-is output for auditory by 
clients. 

[0032] In this case, preferably, while receiving a plurality of said other live streaming 
broadcastin g broadcasts , synthesized image data obtained by the synthesizing 
process for synthesizing image data of said plurality of live streaming 
broadcasting b roadcasts d uring receiving are output for auditory by to the 
clients. 

[0033] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting 
broadcast through a network, characterized in that, w hereby w hile inputting 
camera image data, synthesized image data obtained by the synthesizing 
process for synthesizing the other image data to the camera image data during 
inputting €ffe-is_output to satd -the network fo r auditory by client s auditing bv a 
client . 

[0034] Further, the live streaming broadcasting method of the present invention is 

characterized in that at least either one out of static image data and video image 

data is included in satd -the other image data. 
[0035] Further, the live streaming broadcasting method of the present invention is 

characterized in that text display data input by operation during broadcasting 

flf&-is_included in said -the o ther image data. 
[0036] Further, the live streaming broadcasting method of the present invention is 

characterized in that image data produced on the basis of designated 

information which is for designating image display but not image data are-is 

included in said -thc other image data. 
[0037] Further, the live streaming broadcasting method of the present invention is 

characterized in that plug-in data afe-is.included in said -the o ther image data. 
[0038] Further, the live streaming broadcasting method of the present invention is 

characterized in that said -the s ynthesizing process is an alpha blend process or 
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picture-in-picture process. 
[0039] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadca s ting 
broadcast through a network, characterizod in that whereby text display data 
input by operation during broadcasting afe-is_output to seid -the network for 
auditory listening to by clients. 
[0040] Further, Further, the live streaming broadcasting method of the present 
invention is a live streaming broadcasting method for carrying out a live 
broadcasting broadcast through a network, characterized in that w hereby image 
data correspon ding to the live streaming broadcasting method of the present 
invention is characterized in that image data produced on the basis of 
designated information which is for designating image display but not image 
data efe-is^output to said -the network for auditory listening to by clients. 
[0041] Further, Further, the live streaming broadcasting method of the present 
invention is a live streaming broadcasting method for carrying out a live 
broadcasting b roadcast t hrough a network, characterized in that wherein plug-in 
data aFe-is_output to said-the network for auditory listening to b y clients. 
[0042] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting 
broadcast through a network, characterized in that w herein link-addressed 
information of a^browser on the broadcaster side is output as a script, and the 
link address of the^browser on the clients side is designated on the basis of the 
script of said link addressed information to thereby svdtch the link address 
synchronously with the broadcaster side. 
[0043] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting 
broadcast through a network, characterized in — that — ^whereby p osition 
information displayed on the browser on the broadcaster side is output as a 
script, and a display position of a pointer on the browser on the clients side is 
designated on the basis of the script of said position information to thereby 
associate a display position of the pointer on the clients side with the 
broadcaster side. 

[0044] Further, the live streaming broadcasting method of the present invention is a 
live streaming broadcasting method for carrying out a live broadcasting 
broadcast through a network, characteriged in that whereby i mage data of 
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ifflftge-images depicted by operation of the broadcaster on the browser on the 



[0045] Further, the live streaming broadcasting method of the present invention is 
characterized in that said-the_image data of ifflage -images depicted by operation 
of the broadcaster are synthesized with animation image data to output them to 
sftid -the network. 

[0046] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network; — characterized — by comprising synthesizing 
processing means for executing the synthesizing process in an either live 
streaming method of the present invention, and output means for executing said 
output to said network. 

[0047] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network , characterized by comprising receiving means for 
receiving the other live streaming broadcasting b roadcast through said-the 
network, and output means for outputting image data of said-the live streaming 



listening to by clients. 

[0048] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network , characterized by comprising output means for 
outputting text display data input by operation during broadcasting to satd-the 
network for auditory auditing by clients. 

[0049] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, characterized by comprising output means for 
outputting image data produced on the basis of designated information which is 
for designating image display but not image data to -sai4 -the network for 
auditory auditing b y clients. 

[0050] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, charactorizcd by c omprising output means for 
outputting plug-in data to said network for auditory by clients. 



broadcaster side are output to seid-the_network for auditory listening to b y 
clients. 




-broadcast during receiving to said — the network for auditory 
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[0051] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
method through a network, charactorizod wherein link-addressed information 
of a_browser on the broadcaster side is output as a script, and the link address of 
a_browser on the clients side is designated on the basis of the script of said link 
addressed information to thereby switch the link address synchronously with 
the broadcaster side. 

[0052] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
through a network, choractorizcd in that w herein p osition information displayed 
on the browser on the broadcaster side is output as a script, and a display 
position of a pointer on the browser on the clients side is designated on the 
basis of the script of said position information to thereby associate a display 
position of the pointer on the clients side with the broadcaster side. 

[0053] Further, the live streaming broadcasting apparatus of the present invention is a 
live streaming broadcasting apparatus for carrying out a live broadcasting 
broadcast through a network, characterized by comprising output means for 
outputting image data of image depicted by operation of the broadcaster on the 
browser on the broadcaster side are output to said network for auditory by 
clients. 

[0054] Further, the live streaming broadcasting apparatus of the present invention is 
charactorized by comprising may comprise s ynthesizing means for synthesizing 
said image data of image depicted by operation of the broadcaster with 
animation image data, and said output means outputs image data after 
synthesized by said synthesizing means to said network. 

[0055] Further, the live streaming broadcasting system of the present invention is 
characterized by comprisin | g m ay comprise a live streaming broadcasting 
apparatus of the present invention, and a streaming server for delivering image 
data output from said live streaming broadcasting apparatus to clients. 

[0056] Further, the program of the present invention is a program that can be read by a 
computer, and plural cameral synthesizing process for synthesizing a plurality 
of image data input in apparatus provided with said computer is allowed to be 
executed by said computer, characterized in that switching process for selecting 
camera image data in order to selectively apply a suitable plurality of camera 
image data out of three camera image data or more input in said apparatus, and 
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output process for outputting synthesized image data produced by said plural 
cameral synthesizing process from said apparatus are allowed to be executed in 
that order by said computer. 

[0057] Further, the program of the present invention is a program that can be read by a 
computer, characterized in that synthesizing process in the streaming 
broadcasting method of the present invention and said output to said network 
are allowed to be executed by said computer. 

[0058] Further, the program of the present invention is a program that can be read by a 
computer, characterized in that process for receiving a live streaming 
broadcasting through a network, and process for outputting the live streaming 
broadcasting during receiving to said network for auditory by clients are 
allowed to be executed by said computer. 

[0059] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting text 
display data input by operation during broadcasting of the live streaming 
broadcasting to said network for auditory by clients is allowed to be executed 
by said computer. 

[0060] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is alloNved to 
be executed by said computer, characterized in that process for outputting 
image data produced on the basis of designated information that is for image 
display designation but not image data to said network for auditory by clients is 
allowed to be executed by said computer. 

[0061] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting 
plug-in data to said network for auditpry by clients is allowed to be executed by 
said computer. 

[0062] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting link 
address information of browser on the broadcaster side, designating a link 
address of browser on the clients side on the basis of a script of said link 
address information, and thereby synchronously switching the link address on 
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the clients side to the broadcaster side. 

[0063] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting, as a 
script, position information of a pointer displayed on browser on the 
broadcaster side, designating a display position of the pointer on the browser on 
the clients side on the basis off said position information, and thereby 
associating the display position of the pointer on the clients side with the 
broadcaster side is allowed to be executed by said computer. 

[0064] Further, the program of the present invention is a program that can be read by a 
computer, and the live streaming broadcasting through a network is allowed to 
be executed by said computer, characterized in that process for outputting 
image data of image depicted by operation of the broadcaster on the browser on 
the broadcaster said network for auditory by clients is allowed to be executed 
by said computer. 

[0065] The program of the present invention is a program that can be read by a 

computer is characterized in that process for outputting image data including 

plug-in data to a broadcasting network for auditory by clients is allowed to be 

executed by said computer. 
[0066] Further, the recoding medium of the present invention is characterized in that 

the program of the present invention is recorded. 
[0067] Further, the broadcasting method of the present invention is characterized in 

that image data including plug-in data are output to the broadcasting network 

for auditory by clients. 
[0068] Further, the broadcasting apparatus of the present invention is characterized by 

comprising output means for outputting image data including plug-in data to 

the broadcasting network for auditory by clients. 

« 

BRIEF DESCRIPTION OF THE DRAWINGS 
102121 [FIG 1] A block diagram for explaining a streaming broadcasting method 

according to the embodiment of the present invention. 
FFIG 2] A block dia gram showing an editing device and its peripheral devices 

used for a streaming broadcasting method. 
IFIG 3] A view showing main block structure of a control portion provided in 

the editing device, 
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[FIG 4] A flowchart for explaining a flow of process with respect to image data 

of editing process carried out by the editing device. 
[FIG. 5] A flowchart for explaining a flow of procCvSS with respect to voice data 

out of editing process carried out bv the editing device. 
[FIG 6] A view showing a screen display example of a displav portion of an 

editing device during editing process. 
[FIG 7] A flowchart for explaining a flow of plural camera image synthesizing 

process particularly out of editing process. 
[FIG 8] A flowchart for explaining an example of process in case of carrying 

out sprite process. 

[FIG 9] A flowchart for explaining a flow of process in case of synthesizing 

and outputting live streaming broadcasting receiving from a plurality of other 
streaming servers. 

[FIG 10] A view showing a screen displav example in case of executing 

svncro-browser function and svnchro-pointer function. 
[FIG. 11] A flowchart for explaining the svncro-browser function and 

synchro-pointer function. 
[FIG 12] A view showing a screen displav example in case of executing 

hand-written function, 

[FIG. 13] A flowchart for explaining the hand-written function. 

[FIG 14] A block diagram for explaining a flow of process in a conventional 

live streaming broadcasting. 
[FIG 15 ] A block diagram in case of carrying out live broadcasting using a 

number of broadcasting materials in prior art. 
[FIG 16] A flowchart for explaining a flow of main parts in case of technique 

of FIG 15. 
EFFECT OF THE INVENTION 

[0069] According to the present invention, broadcasting which is high in expression at 

low cost can be executed. 
[0070] Or, broadcasting in novel expression that cannot be obtains so far can be 

realized. 

BEST — MODE — FOR — EMBODYING — 5iiB — INVENTTON nRTATT RD 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0071] The embodiments according to the present invention will be described 
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hereinafter with reference to the drawings. 
[0072] [First Embodiment] 

FIG 1 is the whole block view a flowchart showing various structural elements 
for realizing the streaming broadcasting method according to the present 
embodiment. 

[0073] As shown in FIG 1, in the streaming broadcasting method according to the 
present embodiment, in the editing apparatus (streaming broadcasting 
apparatus) 1 on the broadcaster side, while producing image data and voice 
data by an_editing process, S€ftd -the image data and voice data to be produced, 
that is, image data and voice data after the editing process are continuously 
outputj as broadcasting dataj to a streaming server 3 through a network_2. Here, 
the streaming server 3 at output address is designated in advance by the input of 
an^IP (Internet protocol) by the broadcaster or selecting work. Further, as-the 
network 2, there can be montioncd, for example, the internet a_LAN, a 
communication network of a portable information terminal, and the like. 
Further, as-the editing apparatus 1 , there can be, for example, can take the form 
of. but is not limited to> a general purpose PC (Personal Computer). 

[0074] On the other hand, in a terminal for auditory 4 on the clients side, while 
continuously receiving image data and voice data (broadcasting data) from the 
streaming server 3 through the network 2, they are displayed on a display 
portion of the terminal for auditory 4, and output from a speaker of the terminal 
for auditory 4. 

[0075] Thereby, the clients are able to audit images based on image data from the 
broadcaster side continuously and at real time through the network 2. 

[0076] It is noted that as the T he t erminal for auditory there can be mentioned, for 
example, can take the form of. but is not limited to. a portable information 
terminal apparatus such as a_PDA or as a portable telephone, in addition to a 
general purpose PC. ♦ 

[0077] More concretely, ot A Lthe time of auditor v auditing> for example, the clients get 
access to a home page prepared in advance by the broadcaster side and click 

^broadcasting start button j within the home page to thereby enable (display 

and voice output) e#-broadcasting. Or. broadcasting B roadcasting i s -can also be 
started merolv s imply b y getting access to the home page on the broadcaster 
side. At this time, a streaming layer 82 (including a streaming decoder) is 
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started so that an image display of broadcasting is done within a player screen, 
or an image display is done within a screen of the browser 81. As described, for 
getting access to the home page to do auditing, the broadcaster stores data of 
the home page in advance in a server 5_(a server for the home page separately 
separate from the streaming server 3)-S. 

[0078] It is noted that the other streaming server 6 (FIG 1) for broadcasting is a 
streaming server (for example, for the other broadcaster) in order to perform 
live streaming broadcasting by image data output from apparatus except other 
than t he editing apparatus 1. 

[0079] In the foregoing, transmit/receive of broadcasting data (transmit/receive 

between editing apparatus l->-streaming server 3, and between streaming server 

3 ->-terminal for auditory 4) is carried out by designating transmit/receive ends 

by IP (Internet Protocol). 
[0080] FIG 2 is a block diagram showing the editing apparatus 1 and its peripheral 
apparatus. 

[0081] As shown in FIG 2, camera image data from a plurality (for example, six) of 
cameras 21 afe-is_input ift-into.the editing apparatus on the broadcaster side. It 
is noted that the camera 21 may be a camera for outputting camera image data 
as digital data. It may be one for outputting as analog data. In the case of using 
the camera 21 for outputting as analog data, in the editing apparatus 1, editing 
apparatus (described later) is applied to camera image data to be input after 
A/D conversion. 

[0082] Further, in the editing apparatus 1, voice data from a fcifc ^microphonc i s input, 
or voice data from external voice data outputting apparatus 23 is line-input. It is 
noted that the external voice data outputting apparatus 23 may be, for example, 
CD (Compact Disk) player or MD mini Desk) player. 

[0083] In addition, a video card 24 for processing image data, sound cards 25, 2 for 
processing voice data are inserted into the editing apparatus 1. 

[0084] Further, for example, a head phone (second sound device) as a voice monitor is 
connected to the editing apparatus 1. 

[0085] Further, the editing apparatus is provided with a display portion 12 for 
displaying an operation screen Gl (FIG 6) including a display region of image 
data before editing (source image data) and images (images o be broadcast) 
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after editing, a speaker (first sound device) 13 for outputting, for example, 
voice after editing, an operation portion 14 for carrying out the_editing 
operation, a clock portion 15 for carrying out time checking of time and time 
measuring, and a control portion 11 for carrying out the editing process or 
display control of the display portion 12 according to the operation with respect 
to the operation portion 14, being connected to the network 2. 
[0086] Among them, the T he_display portion 12 comprises, for example. A liquid 
crystal display device or a_display device of a cathode-ray tube system. 
Outputting of display data (image data) to the display portion 12 is carried out 
through a video buffer 24a of the video card 2 ^,for example . 

[0087] Further , outputting of voice data to the speaker 13 is carried out through a 

sound buffer 25a of the sound card 25 , for example . 
[0088] Further, the operation portion 14 is constructed by being provided with a 

keyboard 14a and a mouse 14b , for example, . 

[0089] Further, the control portion 11 is, as shown in FIG 3, constructed by being 
provided with CPU (Central Processing Unit), ROM (Read Only Memory) lib, 
RAM (Random Access Memory) 11c and an input/output interface lldj^ef 
example . 

[0090] Among them, CPU 11a is provided with an operating portion and a control 
portion, and programs stored in the ROM lib are executed to thereby perform 
editing process of broadcasting data (image data and voice data), output process 
of broadcasting data to the network 2, output process of voice data to the head 
phone 27, and operation control of the display portion 12 and the speaker 13. 

[0091] In the ROM (recording medium) lib are stored programs for operation and 
control, and data used for exiting. 

[0092] Concretely, the I Ti^rograms storqii in ROM lib include, for example, an 
editing program 31, a streaming decoder program 32, a streaming encoder 
program 33, and a video decoder program 38. 

[0093] Further, data for editing stored in ROM lib include, for example, static image 
data 34, video image data 35, effeet-sound effect data 36 and music data 37. 
Among them, the static image data 34 is, for example, JPEG; the video image 
data 35 is, for example, AVI or mpeg; the effeet-sound effect d ata 36 is, for 
example, a WAVE file; and the music data 37 is, for example, a WAVE file, 
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mp3, WMAorMIDI. 

[0094] RAM 11c is provided with a work region for CPU 11a. In editing, in 
accordance with the editing program 31, RAM 11c is formed, for example, with 
a capture window 41, a picture buffer (for example, two picture buffers 
comprising a first picture buffer 42 and a second picture buffer), and a main 
picture buffer 44 for temporarily storing image data after all image synthesizing 
processes have been finished. It is noted that the number of the picture buffers 
is the number corresponding to that of image data to be synthesized. That is, if 
the number of image data to be synthesized is 3 or more, the number of picture 
buffers is also 3 or more. 
[0095] In the above-described structure, a live streaming broadcasting system 50 
according to the present embodiment is constructed by the editing apparatus 1, 
camera 21, mike 22, video card 24, sound cards 25, 26, voice apparatus 23, 
head phone 27, streaming server 3 and server 5. 
[0096] In the following, various processes carried out by CPU 11a on the basis of the 

execution of the programs will be described. 
[0097] <Process based on the execution of the video decider program38> 

CPU 11a performs the process for decoding video image data 35 as a video 
decoder 45 (FIG 4) (video decoder process). 
[0098] <Process based on the execution of the video decider program32> 

CPU 11a performs the process for decoding video image data 35 as a video 
decoder 45 (FIG 4) (video decoder process). 
[0099] <Process based on the execution of the streaming decider program33> 

CPU 11a performs the process for decoding live streaming broadcasting data 
received from the other streaming server 6 through the network 2 as a 
streaming decoder 46 (FIG 4) (streaming decoder process). 
[0100] <Process based on the execution of an editing program> 

In the following, there are listed pro^sses performed by CPU 11a based on the 
execution of the editing program. 
[0101] "Capture window producing process" 

Process for producing capture windows 41 (in case of the present embodiment, 
concretely, for example, 6 capture windows 41) so as to correspond to a 
plurality (in case of the present embodiment, concretely, for example, 6) of 
cameras 21 in 1:1. 
[0102] "First switching control process" 
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Process for selecting data for storage to a first buffer picture buffer 42 and a 
second picture buffer 43 out of camera image data received in the capture 
window 41. 

[0103] However, in case where only (one) camera image data from one camera 21 is 
used for editing, the one camera image data is selected for storage in the first 
picture buffer 42. 
[0104] "Picture buffer storage process'* 

Process in which camera storage data selected for storage in the first picture 
buffer by switching control, and camera storage data selected for storage in the 
second picture buffer are temporarily stored in the first picture buffer 42 and 
the second picture buffer 43, respectively. 
[0105] In case where only (one) camera image from one camera 21 is used for exiting, 
the one camera image data is stored in the first picture buffer 42, and camera 
image data is not stored in the second picture buffer. 
[0106] "Plural camera image synthesizing process" 

Plural camera image synthesizing process for synthesizing camera image data 
stored in the first and second picture buffers 42, 43 to produce synthesized 
image data (Step S2 in FIG 4). This plural camera image synthesizing process, 
concretely, include, for example, alpha blend process and picture-in-picture 
process. Among them, the alpha blend process is a process for synthesizing a 
plurality of images in a half-transparent state to synthesize them. For example, 
by using the alpha blend process such that while one transparency of image is 
gradually made higher, the other transparent of image is gradually made lower 
to thereby enable switching between the cameras 21 without difference. Further, 
the picture-in-picture process is a process for displaying the other image in one 
image on a small window, which is able to display images of a plurality of 
cameras 21 simultaneously. 
[0107] It is noted that in case where camera^image data selected by the first switching 
control process (Step SI) is only one, the plural camera image synthesizing 
process is not executed. 
[0108] "Process for producing data for a telop" 

Process for producing display data, as a telop, of text data input by operation of 
the keyboard 14a to insert (synthesize) in camera image at real time (Step S3 in 
FIG4). 

[0109] "Process for producing display data for information" 
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Process for producing, as display data for information, display data on the basis 
of information applied to display designation (for example, time, camera 
position and lap time (in race), score in sport game relay) (Step S3 in FIG4) 
[0110] "Plug-in data producing process" 

Process for producing plug-in data (for example, FLASH animation) (Step S5 
in FIG4). 

[0111] "Static image data obtaining process" 

Process for obtaining static image data 35 selected. 

[0112] "Second switching control process 

Process for selecting, for synthesizing process (Step S7 in FIG 4, described 
later), at least one of image data selected by data producing process for a telop 
(Step S3 in FIG 4), data producing process for information (Step S4 in FIG 4), 
plug-in data processing process (Step S6 in FIG 4), static image data 
obtaining process, video decoder process and trimming decoder process. 

[0113] "Image synthesizing process" 

Process for further synthesizing (Step S7 in FIG 4) image data selected by 
second switching control process, synthesized image data produced by plural 
camera image synthesizing process (Step S2). The image data produced by 
this image synthesizing process is display data of image which is the same as 
that is broadcasted. 

[0114] It is noted that where the plural camera image synthesizing process is not 

executed. In this image synthesizing process, there is carried out process for 

synthesizing camera image data from the first picture buffer 42, and image data 

selected by the second switching control process. 
[0115] "Main picture buffer storage process" 

Process for temporarily storing image data produced by the image synthesizing 

process (Step S7) in the main picture buffer 44. 
[0116] Video buffer storage process". 

Process for storing image data from the main picture buffer 44 in a video buffer 

24a of the video card 24. 
[0117] "Primary buffer storage process for effect sound" 

Process for storing effect sound data 36 selected in a primary buffer 51 for 
effect sound (FIG 5). 
[0118] "Sound effect process" 

Process (Step Sll in FIG 5) for applying sound effect to effect sound data 36 
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selected. 

[0119] "Secondary buffer storage process for effect sound" 

Process for collecting effect sound data 36 after sound effect process (Step 

Sll) to store them in a secondary buffer 52 for effect sound. 
[0120] "Music data decode process" 

Process for decoding selected music data 37 as a decoder 53. 
[0121] "Music data mixer process" 

Process for mixing a plurality of music data 37 decoded by a decoder 53. 
[0122] "Mixer process'* 

Process for mixing effect sound data 36 from a secondary buffer 52 for effect 

sound, voice data from voice apparatus 23, voice data from a mike 22, and 

music data after music data mixer process to thereby produce the dame voice 

data as that is broadcasted (Step S13 in FIG 5). 
[0123] "First sound buffer storage process" 

Process for temporarily storing voice data after mixer process (Step S13) in a 

sound buffer 25a of the sound card 25. 
[0124] "First sound device output process" 

Process for outputting music data stored in the sound buffer 25a to a speaker 

13 as a first sound device. 
[0125] "Mixer process for mixer" 

Process for mixing music data selected for monitor (Step S14 in FIG 4). 
[0126] "Second sound buffer storage process" 

Process for temporarily storing music data after the mixer process for monitor 

(Step S14) in a sound buffer 26a of the sound card 26. 
[0127] "Second sound device output process" 

Process for outputting music data stored in the sound buffer 26a in a head 

phone 27 as a second sound device. 
[0128] "Operating screen display process" 

Process for displaying an operation screen Gl of FIG 1 on the display screen 
of the display portion 12. 
[0129] Here, the function of various display regions formed in the operation screen Gl 

and operating buttons will be described referring to Fig. 6. 
[0130] That is, the operation screen Gl is formed with a display region 61 for carrying 
out image display on the basis of camera image data from any of camera 21 
selected out of a plurality of cameras 21, an operating button 62 for switching 
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camera images display on the display region 61, a display region 63 for 
displaying an image (image based on image data after image synthesizing 
process of Step S7) that is the same as that to be broadcasted or selected plug-in 
data (at the time of selection), a display region 64 for displaying an operating 
window for executing various functions such as telop input, an operating button 
65 for switching various functions using the display region 64, a cross fader 
operating portion 68 for images for carrying out switching between cameras 21, 
an operating button 69 for adding image effects such as picture-in picture, telop 
insertion, static image synthesizing and the like, an operating button 71 for 
selecting effect sound data 36, a display region 72 for displaying a list of 
selection candidate of music data 37, a speaker 13, and a cross fader operating 
portion 73 for voices for adjusting sound amount of he head phone 27. 
[0131] It is noted that the operating buttons 62, 65, 67, 69, 71 can be operated by 
clicking them using a mouse 14b, and the cross fader for images 68 and the 
cross fader operating portion for voices 73can be operated by dragging them 
using a mike 14b. 

[0132] Further, the image data of images displayed on the display region 11 are input 
in the display portion 12 through the video buffer 24a of the video card 24 from 
any of the capture windows 41 selected, and displayed on the basis of the 
image data (In FIG 4, for the simple sake, the video card 24 in a signal channel 
to the display portion 12 from the capture window 41 is omitted.). 

[0133] In the following, an example of concrete operation will be described. 

[0134] <First Operating Example> 

In the first operating example, a description will be made of the case where 
while inputting one image data from one camera 21, synthesized image data 
obtained by synthesizing other image data to said camera image data during 
inputting is output to a network 21 for auditory by clients. 

[0135] In this case, in a first switching control (Step SI), only the camera image data 
received in any one of capture window 41 is selected for storage to a first 
picture buffer 42. Plural camera image synthesizing process S2 is not applied to 
camera image data read from the first picture buffer 42, but said camera image 
data is applied without modification to image synthesizing process (Step ST). 

[0136] On the other hand, in a second switching control (Step S6), at least one image 
data obtained by data producing process for a telop (Step S3), display data 
producing process for information (Step S4), plug-in producing process (Step 
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S5), static image obtaining process, video decoder process, and streaming 
decoder process. 

[0137] Further, in image synthesizing process (Step S7), image data selected by a 
second switching control (Step S6), and image data from a first picture buffer 
42 are synthesized. Thereby, data for display of the same image as that is 
broadcasted is produced. 

[0138] Image data after image synthesizing process is stored in a main picture buffer 
44, and further stored in a video buffer 24a, 

[0139] Image data of the video buffer 24a are output to a display portion 12 for 
monitor, and applied to display in a display region 63 (FIG 6), and are output 
also for encode process by a streaming encoder 47. 

[0140] On the other hand, voice data from voice apparatus or a mike 22, effect sound 
data 36 having sound effect process applied, and at least any of voice data out 
of music data 37 to which decode process is applied are made to the same voice 
data as that are broadcasted by mixer process (Step S13), after which they are 
output for encode process by a streaming encoder 47. 

[0141] In the streaming encoder 47, image data from a video buffer 24a and voice data 
from a sound buffer 25a are encoded for streaming broadcasting, and data 
(broadcasting data) after encoding are output continuously to a network 2. 

[0142] Also, on the clients side, a browser 81 (FIG 1) is started in a terminal for 
auditory 4 to get access to a home page of the broadcaster, and display data of 
the home page is obtained by a server 5 (server for the home page of the 
broadcaster). 

[0143] And, screen display of the home page is started, or fbroadcasting start button J 

formed in a display screen of the home page is clicked to thereby start live 
streaming broadcasting. At that time, in a terminal for auditory 4, a streaming 
player (streaming decoder) 82 is st^rted. The streaming player 82 performs 
image display based on image data received continuously from a streaming 
server 43, and outputs voice data based on voice data received continuously 
from a streaming server 3 from a speaker of a terminal for auditory 4. 
Thereby, clients are able to audit live streaming broadcasting. 
[0144] As described above, according to the first operating example, the clients are 
able to audit images based on synthesized data obtained by synthesizing other 
image data with camera image data. 
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[0145] <Second Operating Examplo 

In the second operating example, a description will be made of the case where 
while inputting a plurality of image data, synthesized image data obtained by 
synthesizing a plurality of image data during inputting is output to a network 
for auditory by clients. 

[0146] In this case, in a first switching control (Step SI), camera image data received 
in any one of capture window 41, and camera image data received in any of 
other capture window 42 are selected for storage to a first picture buffer 42, and 
for storage to a second picture buffer 43, respectively. Further, plural camera 
image synthesizing process (Step S2) is applied to the camera image data read 
from the first and second picture buffers 42, 43 to thereby produce synthesized 
image data. 

[0147] Further, in this case, in a second switching control (Step S6), at least any one of 
image data may be selected, similarly to the case of the first operating example, 
or any one of image data may not be selected. 

[0148] In image synthesizing process (Step S7), in the case of selecting anyone of 
image data in a second switching control, there is carried out process for 
synthesizing the selected image data, and synthesized image data after plural 
camera image synthesized process. On the other hand, in the case of not 
selecting any of image data in the second switching control, the image 
synthesizing process (S7) is not carried out, but synthesized image data after 
plural camera image synthesizing process is stored without modification in a 
main picture buffer 44. 

[0149] Also, in the second operating example, voice process and thereafter image 
process are similar to that of the first operating example. 

[0150] Out of operation in the second operating example, process from image data 
input from camera 21 to plurality camera image synthesizing process (Step S2) 
will be described with reference to a flowchart of FIG 7. 

[0151] First, image data are input from camera 21 and received in capture window 41 
(Step S15). It is noted that where image data from camera 21 is analog data, in 
Step (S15), A/D conversion is applied to image data before receipt into the 
capture window 41. 

[0152] Next, a first switching control process (Step SI) is applied to each image data. 
[0153] Next, camera image data selected in the first switching control process are 
stored in the first and second picture buffers 42, 43 (Steps S16, S17). 
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[0154] Next, the plural camera image synthesizing process (Step S2) is applied to 
image data stored in the first and second picture buffers 42, 43. 

[0155] Further, image data after plural camera image synthesizing process are output to 
a network 2 after having been applied with encode process by a streaming 
encoder 47 through a main picture buffer 44 and a video buffer 24a. 

[0156] As described, according to the second operating example, it is possible to carry 
out process for synthesizing a plurality of camera images, without carrying out 
input/output of image data between a plurality of broadcasting apparatuses or 
repeatedly carrying out A/D conversion and D/A conversion. That is, wasteful 
matter of process in prior art can be eliminated, and there occurs no noise in 
image data due to repeating of A/D conversion and D/A conversion. 

[0157] <Third Operating Examplo 

In the third operating example, concrete operation in case of inputting 
(insertion) by operation during broadcasting will be described. 

[0158] In this case, the broadcaster operates an operating button 65 corresponding to 
telop input to switch display in a display region 64 to an operating window for 
telop input. Thereby, process for producing data for telop (Step S3) becomes 
enabled. 

[0159] In the process for producing data for telop, in the operating window for telop 
input, telop input place is selected, for example, by a mouse pointer, letters are 
input in a frame (text box) for telop input displayed on the selected place by 

operating a keyboard 14a, and a button corresponding to telop display j out 

of operating buttons 69 is clicked. Then, in the second switching control (Step 
86), image data (that is, display data of telop) obtained by the process for 
producing data for telop is selected. 

[0160] In this manner, telop can be inserted into image at real time by editing work 
while executing live streaming broacjpasting. 

[0161] As described, according to the third operating example, since telop can be 
inserted into the image at real time, it is not necessary prepare display data for 
telop in advance to store it, different from the case of prior art (FIG 15), and it 
is possible to carry out insertion of telop in a simple manner.. Further, also in 
the case where telop becomes necessary suddenly, one can correspond thereto 
immediately. 

[0162] <Fourth Operating Example> 
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In the fourth operating example, image data produced on the basis of 
designated information (for, example, time information, camera position 
information, score information in game of sports or the like) that is for image 
designation but not image data is synthesized with camera image data. 

[0163] In this case, for example, when ^watch display button j '"not shown j formed 

in an operating screen Gl is clicked, time information is obtained from a time 

portion 15, and image data for time display is produced on the basis of the 

obtained time information, and the image data is synthesized with camera 

image data to output it for broadcasting. 
[0164] <Fifth Operating Example> 

In the fifth operating example, plug-in data (for example, FLASH animation) is 

synthesized with camera image. 
[0165] In this case, when an operating button 67 corresponding to the desired plug-in 

data is clicked, the plug-in data is synthesized with camera image data to output 

it for broadcasting. 
[0166] <Sixth Operating Example> 

In the sixth operating example, a description will be made of the case where 

sprite process is applied to image data and static image data 34 fi-om camera 21. 
[0167] The sprite process is a process wherein for example, specific color of static 

image data 34 is converted into transparent color, and the static image data 34 

and image data from camera 21 are superposed and synthesized so that display 

priority of the static image data 34 is to be upper level. 
[0168] In this case, .for example, as shown in FIGS, process prior to plural camera 

image synthesizing process (Step S2) is different from the case shown in FIG 

4. 

[0169] That is, image data from camera 21 received in a capture window 41 are 

applied to a third switching control pjocess (Step S21). 
[0170] In the third switching control process, for example, any one of image data, and 

the other image data are selected for storage to the first picture buffer 42, and 

sprite process (Step S23), respectively. 
[0171] On the other hand, in the fourth switching control process (Step S22), for 

example, any one o out of a plurality of static image data is selected in order to 

apply to the sprite process. 
[0172] In the sprite process (Step S23), for example, the sprite process is applied to 
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image data from any one of camera 1, and static image data 34. Image data 
after synthesizing of image data (image data from camera 21) after sprite 
process and static image data 34 are applied to the plural camera image 
synthesizing process (Step S2), which is then synthesized with image data from 
a first picture buffer 42. 

[0173] In the six operating example, clients are able to audit images based on the 
image data applied with the sprite process. 

[0174] <Seventh Operating Example> 

In the seventh operating example^ a description will be made of the case where 
while receiving live streaming broadcasting from the other streaming server 
through a network 2, image data f live streaming broadcasting during receiving 
is output to the network 2 for auditory by clients. 

[0175] In this case, in the second switching control (Step S6), image data after 
streaming decoder process by a streaming decoder 46. 

[0176] As a result, image data of live streaming broadcasting received from the other 
streaming server 6, or synthesized image data obtained by synthesizing the 
other image data with the said image data are respectively output (broadcasted) 
to the network 2. 

[0171] According to the seventh operating example, clients are able to audit images 
using image data of the live streaming broadcasting received from the other 
streaming server 6. 

[0178] <Eighth Operating Example> 

In the case of the eighth operating example, as shown in FIG9 while receiving 
live streaming broadcasting from a plurality of other streaming server 6 
through a network 2, image data obtained by synthesizing process (streaming 
data synthesizing process: Step S3) for synthesizing image data of a plurality 
of live streaming broadcasting during receiving are output to a network 2 for 
auditory by clients. 

[0179] It is noted, in the streaming data synthesizing process, for example, alpha blend 

process or picture-in picture process is carried out. 
[0180] Further, process (Step S32) for synthesizing the other image data (telop, static 

image, video image data or the like) may be applied to synthesized image data 

obtained by the streaming synthesizing process. 
[0181] Synthesized image data after process of Step S31 or Step S32 are output to the 

network 2 while applying encoding by a streaming encoder 47. 
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[0182] As described, according to the eighth operating example, since 
multi-dimensional broadcasting from a plurality of streaming servers 6are 
output to the network 2 for auditory, clients are able to audit multi-dimensional 
broadcasting from a plurality of streaming servers 6. 

[0183] [Second Embodiment] 

In the second embodiment, a description will be made of a synchrony-browser 
function in which link address information of a browser on the broadcaster side 
t output as a script, and a link address of the browser on the clients side is 
designated on the basis of the script of the link address information to thereby 
synchronously switch the link address on the clients side to the broadcaster 
side. 

[0184] FIG 10 is a view showing display on he broadcaster side and the clients side 
during execution of the synchro-browser function. 

[0185] As shown in FIG 10, on a display screen G2 of a display portion 
12of an editing device 1 on the broadcaster side are displayed a 
browser 91, a mouse pointer 92 within the browser 91, and a 
display region 93 for carrying out (that is, display of image to be broadcasted) 
on the basis of image data produced by any of editing process described in the 
first embodiment. 

[0186] On the other hand, on a display screen G3 of a terminal for auditory4 on the 
clients side are displayed a browser 95, a pointer 96 within the browser 95, and 
a display region 97 for carrying out image display on the basis of image data 
broadcasted, It is noted that display data of the pointer 96 is downloaded, at the 
time of getting access to a server 5 for a home page of the broadcaster, is stored 
and held in the terminal for auditory 4 till the browser 95 is terminated, and is 
used for displaying the pointer 96. 

[0187] Next, it is supposed that operation for switching a link address of the browser 
91 is carried out on the broadcaster«side. Then, the editing device 1 converts 
address information of the browser, that is URL (Uniform Resource Locator) to 
script to output it. 

[0188] Then, the terminal for auditory 4 receives script from the editing device 1 
through a network 2 and a streaming server 3, and converts display of the 
browser 95 to a link address designated by the script, 

[0189] Further, in the second embodiment, position information of a mouse pointer 
(pointer) 92 displayed on a browser on the broadcaster side is output as script, 
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and a display position of a pointer 96 on a browser 95 on the auditory side is 
designated on the basis of the script of the position information to thereby 
associate the display position of the pointer 96 on the clients side with the 
pointer 92 on the broadcaster side (synchro-pointer function). 

[0190] That is, the editing device 1 converts position information (coordinate position 
on the browser 91) to script every time when a position of the pointer 92 moves 
on the broadcaster side to output it. 

[0191] Then, the terminal for auditory 4 receives the script from the editing device 1 
through the network 2 and the streaming server 3, and converts the display 
position of the pointer 96 to the position designated by the script (coordinate 
position on the browser 95). 

[0192] Next, these syncro-browser function and synchro-pointer function will be 
described with reference to a flowchart of FIG ll.It is noted one shown in FIG 
11 is process for which a control portion 11 of the editing device 1 is carried 
out. 

[0193] As shown in FIG 11, first, judgment is made whether the synchro-browser 
function is started by operation of the broadcaster (Step S41). 

[0194] In the case where the function is started (YES in Step S41), the coordinate of 
the mouse pointer 92 is converted to the script to perform process for output 
(Step S42, and then link address information of the browser 91 is converted to 
the script for output it (Step S43). 

[0195] In the succeeding step S44, judgment is made whether the synchro-browser 
function is terminated by operation of the broadcaster. 

[0196] In the case where the function is not finished (NO in Step S4), the step moves 
to Step S45. 

[0197] In Step Se45, judgment is made whether the coordinate of a mouse pointer 92 is 
changed, and in the case of judgment in which the coordinate is changed (YES 
in Step S45), process for converting the coordinate of the mouse pointer 92 to 
the script to output it is carried out (Step S46), and the step moves to Step S47. 
On the other hand, in Step S45, in the case of judgment in which the coordinate 
of the mouse pointer 92 is not changed (NO in Step S45), Step S46 is skipped, 
and the step moves to Step S47. 

[0198] Further, In Step S47, judgment is made whether link address (link address 
information) is changed, and in the case of judgment in which the link address 
is changed (YES in Step 47), process for converting link address information of 
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the browser 91 to the script to output it (Step S45), and the step moves to Step 
S44 again. On the other hand, in Step S4, in the case of judgment in which the 
link address is not changed (NO in Step S47), Step S48 is skipped, and the step 
moves to Step S44. 

[0199] Further, in the case where judgment is made in which the synchro-browser is 
finished I Step S44, and in the case where judgment is made in which the 
synchro-browser function is not started in Step S41, the process in FIG 11 is 
finished. 

[0200] According to the second embodiment as described above, since the 
synchro-browser function and synchro-pointer function as described above can 
be realized, for example, presentation, conference or lecture can be suitably 
carried but through the network. At that time, the broadcaster may merely talk 
while touching the browser 91 by the mouse to carry out presentation, 
conference or lecture in a simple manner. 

[0201] Data of small capacity (script of link address information) may merely be 
output for switching display of browser 95 n the clients side, and therefore, data 
capacity handled in the editing device 1 on the broadcaster side can be 
suppressed as small as possible, and broadcasting contents excellent in 
expression can be obtained with less process data amount. 

[0202] In addition, any of broadcasting described in the first embodiment is carried out 
along with the synchro-browser function and synchro-pointer function as 
described above, and therefore, the broadcasting contents can be displayed in a 
display region 97 to enable obtaining broadcasting further excellent in 
expression. For example, in the display region 97, a presenter or a program 
director for conference or lecture is displayed to thereby more easily 
understand presentation, conference or lecture. 

[00203] [Third Embodiment] 

In the third embodiment, a description will be made of an example 
(hand-written function) in which as shown in FIG 12, image data of images 
depicted by operation during broadcasting on a browser 91 on the broadcaster 
side are output to a network 2 for auditory by clients. 

[0204] In this case, as shown in FIG 12, the broadcaster operates an operating portion, 
for example, such as a mouse 14b during broadcasting to provide a depicted 
image on a browser 91, whereby its depicted image is reflected so that the 
image data is synthesized with, for example, animation data (camera image 
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data from camera 21, video image data from a video decoder 45, or image data 
of other streaming broadcasting from a streaming decoder 46) to output it to a 
network 2. 

[0205] As a result, image depicted by operation of the broadcaster is also reflected by 
display on a browser 495 of the auditory terminal 4on the clients side. 

[0206] Next, a flow of process in the case of the third embodiment will be described 
with reference to FIG 13. 

[0207] Animation data 98a is, as described above, for example, camera image data 
from camera 21, video image data from a video decoder 45, or image data of 
other live streaming broadcasting from a streaming decoder 46. Further, image 
data 98b is image data of an image layer in which depicted image by the 
broadcaster is reflected on display. These image data 98b and animation data 
98a are synthesized by synthesizing process 99. As a result, image data after 
synthesizing is data for displaying to which depicted image depicted by the 
broadcaster is superposed. 

[0208] Such image data after synthesizing is stored in a main picture buffer 44, after 
which it is encoded for streaming broadcasting by a streaming encoder 47 and 
output to a network. 

[0209] The terminal for auditory 4 for receiving image data output as described is able 
to audit broadcasting contents in which depicted image by the broadcaster is 
reflected. 

[0210] According to the third embodiment as described, the broadcaster performs 
depicted image in a simple manner at real time to enable causing image display 
on the basis of image data of the depicted image to carry out by the terminal for 
auditory 4. Thereby, presentation can be carried out easily through the network 
2. 

[0211] In the above-described embodiments, a description has been made on 
assumption of streaming broadcasting, but for example, the technique for 
outputting image data including plug-in data for broadcasting may be applied, 
not limiting to the live streaming broadcasting, to other broadcasting methods. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0212] [FIG 1] A block diagram for oxploining a streaming broodcasting method 
according to the embodiment of the present invention. 

[FIG 2] A block diagram showing an editing dcvieo and its peripheral dovices 
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used for Q atroaming broadcasting method. 

[FIG 3] A view showing main block structure of a control portion provided in 

the editing device. 

[FIG. 4 ] A flowchart for explaining a flow of process with respect to image data 

of editing process carried out by the editing device. 
[FIG. 5] A flowchart for explaining a flow of process with respect to voice data 

out of editing process carried out by the editing device. 
[FIG 6] A view showing a screen display example of a display portion of on 

editing device during editing process. 
[FIG 7] A flowchart for explaining a flow of plural camera image synthesizing 

process particularly out of editing process. 
[FIG. 8] A flowchart for explaining an example of process in case of carrying 

out sprite process. 

[FIG Q] A flowchart for explaining a flow of process in case of synthesizing 

and outputting live streaming broadcasting receiving from a plurality of other 
streaming servers. 

[FIG 10] A view showing a screen display example in case of executing 

syncro browser function and synchro pointer function. 
E^€e — 11] A flowchart for explaining the syncro browser function and 

synchro pointer function. 
[FIG 12] A view showing a screen display example in case of executing 

hand written function. 

[FIG 13] A flowchart for explaining the hand written function. 

[FIG 1 4 ] A block diagram for explaining a flow of process in a conventional 

liv e streaming broadcasting, 
[FIG 15] A block diagram in case of carrying out live broadcasting using a 

number of broadcasting materials in prior art> 
[FIG 16] A flowchart for oxplainina a flow of main parts in case of technique 

of FIG 15. 

DESCRIPTION OF REFERENCE NUMERi\LS 

[0213] — 1 Editing device (live streaming broadcasting device) 

2Network 
QlBowsor 
95Brow3cr 

92Mousc pointer (pointer) 
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96PointGr 

ITStrcaming encoder (output means) 

46 Streaming encoder (recomng means) 

Hh ROM (recording medium) 
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